This does appear to fix the image build problem. But I don't know why...can you explain? Why was the version of github.com/prometheus/client_golang from the vendor directory not working?

The default is -mod=vendor, which I think is what we want so that the exact vendor directory is used. The point of checking in the vendor directory is so that it will be used and the build will work even if a dependency gets deleted from github and the go ecosystem. Using -mod=readonly seems like a change in behavior that we did not intend and we could be exposed to dependencies that go away.

From the go doc:

-mod=readonly tells the go command to ignore the vendor directory and to report an error if go.mod needs to be updated.
-mod=vendor tells the go command to use the vendor directory. In this mode, the go command will not use the network or the module cache.

This does appear to fix the image build problem. But I don't know why...can you explain? Why was the version of github.com/prometheus/client_golang from the vendor directory not working?

The default is -mod=vendor, which I think is what we want so that the exact vendor directory is used. The point of checking in the vendor directory is so that it will be used and the build will work even if a dependency gets deleted from github and the go ecosystem. Using -mod=readonly seems like a change in behavior that we did not intend and we could be exposed to dependencies that go away.

From the go doc:

-mod=readonly tells the go command to ignore the vendor directory and to report an error if go.mod needs to be updated.
-mod=vendor tells the go command to use the vendor directory. In this mode, the go command will not use the network or the module cache.

I don't know, I'll put the pr in draft.
I'll be thinking and looking.

This does appear to fix the image build problem. But I don't know why...can you explain? Why was the version of github.com/prometheus/client_golang from the vendor directory not working?

The default is -mod=vendor, which I think is what we want so that the exact vendor directory is used. The point of checking in the vendor directory is so that it will be used and the build will work even if a dependency gets deleted from github and the go ecosystem. Using -mod=readonly seems like a change in behavior that we did not intend and we could be exposed to dependencies that go away.

From the go doc:

-mod=readonly tells the go command to ignore the vendor directory and to report an error if go.mod needs to be updated.
-mod=vendor tells the go command to use the vendor directory. In this mode, the go command will not use the network or the module cache.

the error was here - .dockerignore

This does appear to fix the image build problem. But I don't know why...can you explain? Why was the version of github.com/prometheus/client_golang from the vendor directory not working?
The default is -mod=vendor, which I think is what we want so that the exact vendor directory is used. The point of checking in the vendor directory is so that it will be used and the build will work even if a dependency gets deleted from github and the go ecosystem. Using -mod=readonly seems like a change in behavior that we did not intend and we could be exposed to dependencies that go away.
From the go doc:

-mod=readonly tells the go command to ignore the vendor directory and to report an error if go.mod needs to be updated.
-mod=vendor tells the go command to use the vendor directory. In this mode, the go command will not use the network or the module cache.

the error was here - .dockerignore

Ah yes... I see how *test was the problem since it matches latest, good find!

I think you don't need the two file renames anymore, since they don't match *_test, right?

the change indeed solves the issue, while since @denyeart had concerns, will wait for his decision.